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We search for B~ — > D^K~ and B~ — > D*°K~ and charge conjugates. Here the symbol D° 

indicates decay of a or into K^it~ , while the symbol I)*" indicates decay of a D*" or D*" 
to D'^tt" or D"'y. These final states can be reached through the b ^ c transition B^ D^*^'^K~ 
followed by the doubly Cabibbo-Kobayashi-Maskawa (CKM)-suppressed K^tv~ , or the b ^ u 

transition B~ d'-*'>°K~ followed by the CKM-favored D° -> JT+tt". The interference of these 
two amplitudes is sensitive to the angle 7 of the unitarity triangle. Our results axe based on 232 
million T{4S) BB decays collected with the BABAR detector at SLAC. We find no significant 
evidence for these decays. We set a limit rs = \A{B- WK-)/A{B- D^K-)\ < 0.23 at 
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90% C.L. using the most conservative assumptions on the values of the CKM angle 7 and the 
strong phases in the B and D decay amplitudes. In the case of the D* we set a 90% C.L. limit 
Tfl = \A{B~ D*°K')/A{B' ^ D*°K~)\'^ < (0.16)^ which is independent of assumptions on 7 
and strong phases. 

PACS numbers: 13.25.Hw, 14.40.Nd 




I. INTRODUCTION 

Following the discovery of CP violation in B-meson de- 
cays and the measurement of the angle (3 of the unitar- 
ity triangle [1] associated with the Cabibbo-Kobayashi- 
Maskawa (CKM) quark mixing matrix, focus has turned 
towards the measurements of the other angles a and 
7. The angle 7 is aTg{-V*i,V^JV*t,V^^), where V,^ are 
CKM matrix elements. In the Wolfenstein convention [2], 
7 = arg(y„*J. 

Several proposed methods for measuring 7 exploit the 
interference between B~ — > and B~ 

(pjg^ ^T^g^^ occurs when the D^*)" and the 
D^*^'^ decay to common final states, as first suggested in 
Ref. [3]. 



B- 



FIG. 1: Feynman diagrams for B~ — > D(*)''7f(*)~ and 

£)(*)o^(*)- rpjjg latter is CKM and color suppressed with 

respect to the former. The CKM-suppression factor is 

|KibVcs/''4i)K^al ~ 0.4. The naive color-suppression factor is 
i_ 

3' 



As proposed in Ref. [4], we search for B D^K 
and B- ^ D*°K-, D*° D°t:° or D°-f, fol- 

lowed by K^TT^, as well as the charge conjugate 

(c.c.) sequences. Here the symbol £>" indicates the de- 
cay of a D'-' or 1)° into if+7r~. In these processes, the 
favored B decay (B^ D^*^"K^) followed by the dou- 
bly CKM-suppressed D decay K~^tt~) interferes 
with the suppressed B decay {B~ D^*^°K~) followed 
by the CKM-favored D decay {D° K+tt-). The in- 
terference of the b ^ c {B~ — > D^*'^°K^) and b ^ u 
{B~ D'^*^^K^) amplitudes is sensitive to the relative 
weak phase 7. 

We use the notation B^ [/ij^/ij^Jj^/ig" (with each 
hi — TT or K) for the decay chain , 
K^h^ ■ For the closely related modes with a we use 
the same notation with the subscript D replaced by Dtt" 



or 1)7, depending on whether the Z)*" decays to D^tt^ or 
£'°7. We also refer to as the bachelor it or K. 

In the decays of interest, the sign of the bachelor kaon 
is opposite to that of the kaon from I?" decay. It is 
convenient to define ratios of rates between these de- 
cays and the similar decays where the two kaons have 
the same sign. The decays with same-sign kaons have 
much higher rate and proceed almost exclusively through 
the CKM-favored and color-favored B transition, fol- 
lowed by the CKM-favored D-decay, e.g., B^ D°K^, 
— + K~'K^ . The advantage of taking ratios is that 
most theoretical and experimental uncertainties cancel. 
Thus, ignoring possible small effects due to D mixing and 
taking into account the effective phase difference of tt be- 
tween the D* decays in and Dtt" [5], we define the 
charge-specific ratios for D and D* as: 



(1) 



7^ 



= r*i +rl + 2rlrDCOB{±-f + 5*), (2) 



and 



7^ 



*± 



T{[K^t:^]d^K^) 



= r*B^ + rl-2r*srDCOsi±j + S*), (3) 



where 



A{B- - 


-> D^K-) 


A{B- - 


D^K-) 


A{B- - 




A{B- - 


^ D*OK-) 


A{D^ - 


K+TT-) 


A{D^ - 


K-TT+) 



si*) 



(4) 



(5) 



(6) 



(7) 



*Also with Universita della Basilicata, Potenza, Italy 
t Deceased 



and 6^ and 6d are strong phase differences between the 
two B and D decay amplitudes, respectively. The value 
of has been measured to be td = 0.060 ± 0.002 [6]. 
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We also define the charge-independent ratio 

^ T{[K+tt-]dK-) + Ti[K~7T+]DK+) 
K-K. - y{[K-t,+]dK-)+T{[K+t,-]dK+) 

and the equivalent ratios for the D* modes, 

_ T{[K+^-]D^oK-)+T{[K-n+]D^oK+) 



As wc will describe below, the measured 7?,}^^ are con- 



T{[K-Tr+]D^oK-) + r{[K+Tr-]D^oK+) ' 



and 



(9) 



Then, 



T{[K-n+]njK-) + T{[K+n-]D^K+) ' 



7^+ _)_ 7^- 

— f^2_ — !<JL = r%+r% + 2rBrD cos-f cos 6 (11) 



and, similarly for the D* modes, 

'^*Kn,DnO = rf + rjj + 2r*BrD cos 7 cos S* , (12) 



^K7r,r>7 = ''b + ''i? - 2rBrr, cos 7 cos (J*. 



(13) 



Equations 11, 12, and 13 assume no CP violation in the 
normalization modes [K^tt^]dK^ , [K^Tr^]]jT^oK^ , and 
[K'^TT^jD-yK^ . In the following we use the notation TZ'^^ 
when there is no need to refer specifically to TZ%;^ j^^o or 

As discussed below, the parameter r^' is expected to 
be of the same order as rjj. Thus. CP violation could 

manifest itself as a large difference between the charge- 

(*)+ (*)— 
specific ratios TZj^!^ and 1^)^^ ■ Measurements of these 

six ratios can be used to constrain 7. 

The value of determines, in part, the level of in- 
terference between the diagrams of Fig. 1. In most tech- 
niques for measuring 7, high values of lead to larger 
interference and better sensitivity to 7. Thus, rs and 
are key quantities whose values have a significant im- 
pact on the ability to measure the CKM angle 7 at the 
B-factories and beyond. 

In the standard model, r'^*^ = K,V,%/VM Fes ~ 
QAFcs- The color-suppression factor Fes < 1 accounts 
for the additional suppression, beyond that due to CKM 
factors, of B' D^*^°K- relative to ^ D^*^°K-. 
Naively, F,,s = I, which is the probability for the color 
of the quarks from the virtual W in B~ D^*^'^K~ to 
match that of the other two quarks; see Fig. 1. Early es- 
timates [7] of Fes were based on factorization and the ex- 
perimental information on a number of 6 ^ c transitions 
available at the time. These estimates gave Fes ~ 0.22, 
leading to w 0.09. However, the recent observations 
and measiircments [8] of color-suppressed b ^ c decays 
{B D'-*^h^; hP = 7r°, ;9°, w, 77, 77') suggest that color 
suppression is not as effective as anticipated, and there- 
fore the value of r^' could be of order 0.2 [9]. 



sistcnt with zero in the current analysis. Since TZ)^!^ de- 
(8) pend quadratically on r^'' , we will use our measurements 



to set restrictive upper limits on ' . 

It is important to note the different signs of the third 

terms in the expressions for 7?.^^ ^^^o and TZ}^^ in 
Equations 12 and 13. This relative minus sign is due 
to the phase of tt between the two D* decay modes [5]. 
It allows for a measurement of with no additional in- 
puts since r^^ = II^DttO ^" ^1)7) ~ ^i)' ^^'^ is known 
quite precisely (ru = 0.060 ± 0.002). We will use this 
equation for and our results for TZ'^^o and TZ})^ to set 
an upper limit on with no assumptions. 

On the other hand, TZk-k depends on the three un- 
knowns rs, 7, and S, see Equation 11; thus, in order to 
extract a limit on Vb from the experimental limit on TZktt 
we must make assumptions about 7 and 5. As we will 
discuss in Section IV, we have chosen to quote an upper 
limit on tb based on the most conservative assumptions 
on 7 and 5. 

In this paper we report on an update of our previ- 
ous search for B^ D^K~ [10], and the first attempt 
to study B~ D*°K~. The previous analysis was 
based on a sample of i?-meson decays a factor of 1.9 
smaller than used here, and resulted in an upper limit 
TZk^ < 0.026 at the 90% C.L. This in turn was trans- 
lated into a limit tb < 0.22, also at the 90% confidence 
level. A similar analysis by the Belle Collaboration [11] 
gives limits TZk^k < 0.044 and rs < 0.27 (90% C.L.). 

Information on rs, r%, and 7 can also be obtained 
from studies of B± D^K^ and B± -> D*"K^, 
D° KsTT~^7T~. An analysis by the Belle collabora- 
tion [12] finds quite large values = 0.31 ± 0.11 and 

= 0.34 ± 0.14, although the imcertainties arc large 
enough that these results are not inconsistent with the 
limits listed above. On the other hand, a similar analysis 
by the BABAR Collaboration [13] favors smaller values for 
these amplitude ratios: rs = 0.12±0.09 at 90% C.L. and 
r*o = 0.17 ±0.10. 



II. THE BABAR DATASET 



The results presented in this paper are based on 232 x 
lO*' T{4S) BB decays, corresponding to an integrated 
luminosity of 211 fb~^. The data were collected between 
1999 and 2004 with the BABAR detector [14] at the PEP- 
II B Factory at SLAC [15]. In addition, a 16 fb^^ off- 
resonance data sample, with center-of-mass (CM) energy 
40 MeV below the T(45) resonance, is used to study 
backgrounds from continuum events, e+e~ — * 99 (? = 
u, d, s, or c). 
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TABLE I: Notation used in the text for the decay modes that define the data samples used in the analysis. 



Abbreviation 


Mode 


Comments 


DK 


B- 


D"if-, D" ^ K-TT+ and c.c. 




normalization 


Dtt 


B- 


D°n-, D° K--K+ and c.c. 




control 


DK 


B- -> 


b°K-, b° K+Tv- and c.c. 




signal 


D*K 


B- 


D*°K-, D*° D"n"/j, D° K 


it'^ and c.c. 


normalization 


D*TV 


B- 




tt"*" and c.c. 


control 


D*K 


B- 


D*°K-,D*° ^D°ny'y,D°^ 


K^'K~ and c.c. 


signal 



III. ANALYSIS METHOD 

This work is an extension of our analysis from Ref. [10], 
which resulted in 90% C.L. limits on TZkv < 0.026 and 
rs < 0.22, as mentioned in Section I. The main changes 
in the analysis are the following: 

• The size of the dataset is increased from 120 to 
232 X 10^' r(45) -> BB decays. 

• This analysis also includes the B"^ D*^K'^ 
mode. 

• The event selection criteria have been made more 
stringent in order to reduce backgrounds further. 

• A few of the selection criteria in the previous anal- 
ysis resulted in small differences in the efficiencies 
of the signal mode — > [K^-k'^\dK'^ and the 
normalization mode B^ tt^]dK^. These 
selection criteria have now been removed. 

The analysis makes use of several samples from different 
decay modes. Throughout the following discussion we 
will refer to these modes using abbreviations that are 
summarized in Table I. 

The event selection is developed from studies of simu- 
lated BB and continuum events, and off-resonance data. 
A large on-resonance control sample of Dn and D*Tr 
events is used to validate several aspects of the simu- 
lation and analysis procedure. 

The analysis strategy is the following: 

1. The goal is to measure or set limits on the charge- 
independent ratios TZktt and T^^n- 

2. The first step consists of the application of a set of 
basic selection criteria to select possible candidate 
events, see Section III A. 

3. After the basic selection criteria, the backgrounds 
are dominantly from continuum. These arc signif- 
icantly reduced with a neural network designed to 
distinguish between BB and continuum events, see 
Section IIIB. 

4. After the neural network requirement, events 
are characterized by two kinematical variables 
that are customarily used when reconstructing 
i?-meson decays at the T{4S). These vari- 
ables are the energy-substituted mass, toes = 



^(f + Pa ■ PbY /Eq — P% and energy difference 

AE = Eg — ■^^/s, where E and p are energy and 
momentum, the asterisk denotes the CM frame, the 
subscripts and B refer to the T{4S) and B can- 
didate, respectively, and s is the square of the CM 
energy. For signal events toes = ™-B and AE = 
within the resolution of about 2.5 and 20 MeV, re- 
spectively (here tub is the known B mass [6]). 

5. We then perform simultaneous unbinned maximum 

likelihood fits to the final signal samples (DK and 
D*K), the normalization samples (DK and D*K), 
and the control samples {Dtt and £>*7r) to extract 
TZkit and Tl"^^, sec Section III C. The fits arc based 
on the reconstructed values of toes and AE in the 
various event samples. 

6. Throughout the whole analysis chain, care is taken 
to treat the signal, normalization, and control sam- 
ples in a consistent manner. 



A. Basic Selection Criteria 

Charged kaon and pion candidates in the decay modes 

of interest must satisfy K or ir identification criteria [16] 
that are typically 85% efficient, depending on momentum 
and polar angle. The misidentification rates are at the 
few percent level. 

The invariant mass of the Kw pair must be within 18.8 
MeV (2.5(t) of the mean reconstructed mass (1863.3 
MeV). For modes with D*° I)°7r° and D*° D°-f the 
mass difference AM between the I)*" and the I)" must 
be within 3.5 MeV (3.5ct) and 13 MeV {2a), respectively, 
of the expectation for D*" decays (142.2 MeV). 

A major background arises from DK and D*K decays 
in which the K and tt in the D decay are misidentified 
as TT and K, respectively. When this happens, the decay 
could be reconstructed as a DK or D*K signal event. To 
eliminate this background, we recompute the invariant 
mass (Mswitch) of the pair in £>° h'^h~ switching 

the particle identification assumptions (tt vs. K) on the 
and the h~ . We veto any candidates with Mgwitch 
within 18 MeV of the known D mass [6]; this requirement 
is 90% efficient on signal decays. In the case of D^K, we 
also veto any candidate for which the D*^ is consistent 
with D*+ D°TT+ or D*° D^n^ decay. 
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FIG. 2: Distributions of the continuum suppression neural network (A^A'' and NN') outputs for the three modes. Figs, (a-c) 
show the expected distribution from signal events. The dashed line histogram shows the distribution of simulated signal events 
and the histogram with error bars shows the distribution of D^*^°-k control sample events with background subtracted using the 
rriES sideband (5.20 GeV < mss < 5.27 GeV). Figs, (d-f) show the expected distribution for continuum background events. 
The dashed line histogram shows the distribution of simulated continuum events and the histogram with errors shows the 
distribution of off-resonance events. The rriBS a^nd AE requirements on the off-resonance and continuum Monte Carlo events 
have been kept loose to increase the statistics. Each Monte Carlo histogram is normalized to the area of the corresponding 
data histogram. 



B. Neural Network 

After these initial selection criteria, backgrounds 
are overwhelmingly from continuum events, especially 
e+e- cc, with c D°X, D° ^ K+tt- and c DX, 
D -> K-Y. 

The continuum background is reduced using a neural 
network technique. The neural network algorithms used 
for the modes with and without a D* are slightly differ- 
ent. First, for all modes we use a common neural network 
(NN) based on nine quantities, listed below, that distin- 
guish between continuum and BB events. Then, for the 
modes with a D* we also take advantage of the fact that 
the signal is distributed as cos^ 6d' for D* Dir or 
sin^ 9d* for D* Dj, while the background is roughly 
independent of cos6'd*. Here 9d* is the decay angle of 
the D*, i.e., the angle between the direction of the D 
and the line of flight of the D* relative to the parent B, 
evaluated in the D* rest frame. Thus, we construct a 
second neural network, NN', which takes as inputs the 
output of NN and the value of cos 6'£i. . We then use as 



a selection requirement the output of NN in the D'^K 
analysis and the output of NN' in the D*'^K analysis. 

The nine variables used in defining NN are the follow- 
ing: 

1. A Fisher discriminant [17] constructed from the 
quantities Lq = J^i Pi ^^nd L2 = J2i Pi '^'^'^^ ^i calcu- 
lated in the CM frame. Here, pi is the momentum 
and 6i the angle with respect to the thrust axis of 
the B candidate of tracks and clusters not used to 
reconstruct the B meson. 

2. |cos0t|, where dx is the angle in the CM frame 
between the thrust axes of the B candidate and 
the detected remainder of the event. The distribu- 
tion of I cos 9t I is approximately flat for signal and 
strongly peaked at one for continuum background. 

3. cos 6*3, where 9g is the polar angle of the B candi- 
date in the CM frame. In this variable, the signal 
follows a sin^ 9g distribution, while the background 
is approximately uniform. 
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4. cos 9^ where 6^ is the decay angle in — > Kw. 

5. cos^g'*\ where ^s**' is the decay angle in i? — + 
D^K or B ^ b*^K. In signal events the distri- 
butions of cos^l^ and cos 6*^* ' are uniform. On 
the other hand, the corresponding distributions in 
combinatorial background events tend to show ac- 
cumulations of events near the extreme values. 

6. The charge difference AQ between the sum of the 
charges of tracks in the or £)*" hemisphere and 
the sum of the charges of the tracks in the oppo- 
site hemisphere excluding those tracks used in the 
reconstructed B. The partitioning of the event in 
the two hemispheres is done in the CM frame. For 
signal, (AQ) = 0, whereas for the cc background 
(AQ) « I X Qb, where Qb is the charge of the 
B candidate. The value of (AQ) in cc events is a 
consequence of the correlation between the charge 
of the c (or c) in a given hemisphere and the sum 
of the charges of all particles in that hemisphere. 
Since the AQ RMS is 2.4, this variable provides 
approximately a la separation between signal and 
cc background. 

7- Qb ■ Qk, where Qk is the sum of the charges of 
all kaons not in the reconstructed B, and Qb, as 
defined above, is the charge of the reconstructed B 
candidate. In many signal events, there is a charged 
kaon among the decay products of the other B in 
the event. The charge of this kaon tends to be 
highly correlated with the charge of the B. Thus, 
signal events tend to have Qb ■ Qk < —1- On 
the other hand, most continuum events have no 
kaons outside of the reconstructed B, and therefore 
Qk = 0. 

8. The distance of closest approach between the bach- 
elor track and the trajectory of the D*^. This is 
consistent with zero for signal events, but can be 
larger in cc events. 

9. A quantity A4Ke defined to be zero if there are no 
leptons (e or /i) in the event, one if there is a lepton 
in the event and the invariant mass (niKe) of this 
lepton and the bachelor kaon is less than the mass 
of the Z?- meson (mu), and two if mne > mu- This 
quantity differentiates between continuum and sig- 
nal events because the probability of finding a lep- 
ton in a continuum event is smaller than in a BB 
event. Furthermore, a large fraction of leptons in 
cc events are from D Kiv, where K is recon- 
structed as the bachelor kaon. For these events 
rriKi < rno, while in signal events the expected 
distribution of nixt extends to larger values. 

The neural networks (A'^A^ and NN') are trained with 
simulated continuum and signal events. The distribu- 
tions of the A^ A^ and A^A' outputs for the control sam- 
ples {Dtt, D*^, and off resonance data) are compared 



with expectations from the Monte Carlo simulation in 
Fig. 2. The agreement is satisfactory. We have also ex- 
amined the distributions of all variables used in A^A^ and 
A^A^', and found good agreement between the simulation 
and the data control samples. 

Our final event selection requirement is A^A^ > 0.5 
for DK and A^A^' > 0.5 for D*K. In addition, to re- 
duce the remaining BB backgrounds, we also require 
cos 9b > —0.75. These final requirements are about 40% 
efficient on simulated signal events, and reject 98.5% of 
the continuum background. 

The overall reconstruction efficiencies, estimated from 
Monte Carlo simulation, are about 14% for DK, 8% for 
D*K with D* Dw^ and 7% for D*K with D* D-f. 
Note that a precise knowledge of the efficiencies is not 
needed in the analysis. We apply the identical require- 
ments to the normalization modes DK and D*K. Then, 
in the extraction of TZk-r and T^^w' efficiencies of the 
overall selection cancel in the ratio. 



C. Fitting for event yields and TZj^!^ 

The ratios TZk-r and TZ'^^ are extracted from the ra- 
tios of the event yields in the tues distribution for the 
signal modes (DK and D* K) and the normalization 
modes {DK and D*K), taking into account potential 
differences in efficiencies and backgrounds. All events 
must satisfy the requirements disciisscd above and have 
a AE value consistent with zero within the resolution 
(-52 MeV < AE < 44 MeV). Here we discuss the proce- 
dure to extract TZk-k] the values of TZ"^^ and TZ'^j^ jj^ 
are obtained in the same way. 

The ttzes distributions for DK (signal mode) and DK 
(normalization mode) are fitted simultaneously. The fit 
parameter TZktt is given by TZktt = c- N^jj^/Ndk, where 
N^j^ and Nbk are the fitted yields of DK and DK 
events, and c is a correction factor, determined from 
Monte Carlo, for the ratio of efficiencies between the 
two modes. We find that this factor c is consistent with 
unity within the statistical accuracy of the simulation, 
c = 0.98±0.04 (these correction factors are c = 0.97±0.05 
and c = 0.99 ± 0.05 for D* Dw^ and D* D-y, re- 
spectively) . 

The rriES distributions are modeled as the sum of a 
threshold combinatorial background function [18] and a 
Gaussian lincshape centered at ms. The parameters of 
the background function for the signal mode are con- 
strained by a simultaneous fit of the tties distribution 
for events in the sideband of AE (-120 MeV < AE < 
200 MeV, excluding the AE signal region defined above). 
The parameters of the Gaussian for the signal and nor- 
malization modes are taken to be identical. The num- 
ber of events in the Gaussian is Nsig + Np^ak, where 
A^sig = Nbk or A^/j^ and Apeafc is the number of back- 
ground events expected to be distributed in the same way 
as DK or DK in ttjes ("peaking backgrounds"). 

There are two classes of peaking background events: 
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TABLE II: Summary of fit results. 



Mode 


DK 


D'K, D* Dtv° 


D*K, D* ^ Dj 


Ratio of rates, TZktv or TZ}^^, xlO~^ 

Number of signal events 

Number of normalization events 

Number of peaking charmless events 

Number of peaking D'*'7r events in signal sample 


TZ-Ktv = 13+9^ 
r+4, 

368 ± 26 

'J- ' -0.7 

0.48 ±0.05 


-r.. _ o+iU 

-0 2+1'^ 
150 ± 17 
0+°-^ 
0.18 ±0.03 


7-.* 

1 2+2-^ 
108 ± 14 

0.01 ± 0.02 




A E (GeV) A E (GeV) A E (GeV) 

FIG. 3: AE distributions for normalization events {DK and D* K) with thes within 3(7 oi niB with the fit model overlaid, 
(a) DK events, (b) D* K events with D* D-k^ . (c) D* K events with D* Dj. The dashed (dot-dashed) lines are the 
contributions from Dtt or D*-k {DK or D* K) events. The dotted lines are the contributions from other backgrounds, and the 
solid line is the total. 



1. Charmless B decays, e.g., K^-k^ . These 
are indistinguishable from the DK signal if the 
if~7r+ pair happens to be consistent with the D 
mass. 

2. Events of the type B^ D'^tt^ , where the bach- 
elor TT~ is misidentified as a K~ . When the 
decays into K^tt^ {K'^tt^), these events are indis- 
tinguishable in Toes from DK {DK), since toes is 
independent of particle identification assumptions. 

The amount of peaking charmless B background (1) is 
estimated directly from the data by performing a simul- 
taneous fit to events in the sideband of the reconstructed 
D mass. In this fit the number of charmless background 
events is constrained to be > 0. 

The AE distribution of the Dtt background (2) is 
shifted by about +50 MeV due to the misidentification of 
the bachelor tt a.s a. K. Since the AE resolution is of or- 
der 20 MeV, the AE requirement does not eliminate this 
background completely. The remaining Dtt background 
after the AE requirement is estimated relaxing the AE 
requirement and performing a fit to the AE distribution 
of the DK candidate sample, as described below. 

We fit the AE distribution of DK candidates, with 
Toes within 3(t of m^, to the sum of a DK component, 
a Dtt background component, and a combinatorial back- 
ground component, see Fig. 3. From this fit we can es- 
timate the number of Dtt background events after the 
AE requirement, which we denote as N'^j^. In this fit. 



the AE shapes of the DK and Dtt components are con- 
strained from the data as follows: 

• The large Dtt sample, with the bachelor track iden- 
tified as a pion, is used to constrain the shape of the 
DK component in the sample of DK candidates. 

• The same sample of Dtt events, but reconstructed 
as DK events, is used to constrain the shape of the 
Dtt background in the DK sample. 

The Dtt peaking background is much more important 
in the DK (normalization) channel than in the DK (sig- 
nal) channel. This is because in order to contribute to 
the signal channel, the has to decay into K'^tt~, 
and this mode is doubly CKM suppressed. For the 
DK (signal) sample, the contribution from the residual 
Dtt peaking background in the toes fit is estimated as 
N^^ = rjjN^i^, where ro = 0.060 ± 0.002 is the ra- 
tio of the doubly CKM-suppressed to the CKM-favored 
D Ktt amplitudes and Nf^j^ was defined above. 

The complete procedure simultaneously fits seven dis- 
tributions: the mEs distributions of DK and DK, the 
DK distributions in sidebands of AE and m{D'^), the 
AE distribution of DK, and the AE distributions of Dtt 
reconstructed as Dtt and as DK. All fits are unbinned 
extended maximum likelihood fits. They are configured 
in such a way that TZkti and 7?.^^ are explicit fit pa- 
rameters. The advantage of this approach is that all un- 
certainties, including the uncertainties in the PDFs and 
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itIes (GeV) itIes (GeV) m^^ (GeV) 

FIG. 4: rriES distributions for normalization events {DK and D* K) with Ai5 in the signal region with the fit model overlaid, (a) 
DK events, (b) D* K events with D* — > Dtt". (c) D* K events with D* D'y. The dashed lines represent the backgrounds; 
these are mostly from Dtt or D*tt, and also peak at the _B-mass. As explained in the text, the size of the Dtt and D*n 
backgrounds is constrained by the simultaneous fits to the distributions of Fig. 3. 




> 



IT) 
fN 



> 



(b) 



5.25 

m, 



5.3 
Es(GeV) 



> 



IT) 



s 

> 



(c) 



5.25 
m 



5.3 
Es(GeV) 



FIG. 5: rriEs distributions for DK and D* K events with Kn mass in a sideband of the reconstructed D mass and with AE in 
the signal region. These events are used to constrain the size of possible peaking backgrounds from charmless B-meson decays, 
i.e., decays without a D in the final state. The fit model is overlaid, (a) DK events, (b) D* K events with D* — > Dtt". (c) 
D*K events with D* D'y. Note that the Kn mass range in the sideband selection is a factor of 2.7 larger than in the signal 
selection. 



the uncertainties in the background subtractions, are cor- 
rectly propagated in the statistical uncertainty reported 
by the fit. 

The fit is performed separately for DK, D*K, D* 
Dtt^, and D*K, D* and is identical for all three 

modes, except in the choice of parameterization for some 
signal and background components in the AE fits. 

Systematic uncertainties in the detector efficiency can- 
cel in the ratio. This cancellation has been verified by 
studies of simulated events, with a statistical precision of 
a few percent. The likelihood includes a Gaussian uncer- 
tainty term for this cancellation which is set by the statis- 
tical accuracy of the simulation. Other systematic uncer- 
tainties, e.g., the uncertainty in the parameter rn used to 
estimate the amount of peaking backgrounds from D(*^7r, 
are also included in the formulation of the likelihood. 

The fit procedure has been extensively tested on sets 
of simulated events. It was found to provide an unbiased 
estimation of the parameters TZkjt and 7?,^^. 



IV. RESULTS 

The results of the fits are displayed in Table II and 
Figs. 3, 4, 5, and 6. As is apparent from Fig. 6, we see 
no evidence for the D*K and DK modes. 

For the DK mode we find TZk^t = (13t") x 10"^ 
for the D*K mode we find 7^;^^^^o = (-2te°) x lO^^ 
(for D* DTr°) and 11*^^ 0.^ = (H^is) x 10"^ (for 
D*-^Dj). 

Next, we use our measurements to extract information 
on and r^. In the case of decays into D*" we start 
from equations 12 and 13 to derive 

,*^2^ '^K.,D.o+T^K.,D, (14) 

We use the relationship given by equation 14 in conjunc- 
tion with rjj = 0.060±0.002 and our results for 'R-*^^^ 

and TZ*j^^ to extract information on r*^ with no as- 
sumptions on the values of 7 and strong phases. 

Since the uncertainties in 7?.^^ and 7?,J^^ are 
non-Gaussian, care must be taken in propagating them 
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FIG. 6: niES distributions for candidate signal events with the fit model overlaid, (a) DK events, (b) D*K events with 
D* Dn°. (c) D'K events with D* D-y. 



into an uncertainty in r*^. We interpret the fit likeli- 
hoods for 7?.^^^^Q and TVj^^ j^^ (see Figure 7) as pos- 
terior PDFs assuming constant priors. We assume a 
Gaussian PDF for rn- We then convolve numerically 
the three PDFs for TV^^jy^oj T^Kjr.D-yy and td according 
to equation 14 to obtain a PDF for r'^ which is shown 
in Figure 8. The convolution relies on the fact that the 
measurements of TZ}^^ jj^o and TZ*xt^ are uncorrelated 
(the correlation due to the uncertainty in ro, which was 
used to extract the Dtt peaking backgrounds in the two 
modes, is negligible). 

Our resuh is r*j^ = (41^^^) x 10"^. Based on the PDF 
for r'^ shown in Figure 8 we set an upper limit r'^ < 
(0.16)^ at the 90% C.L. using a Bayesian method with a 
uniform prior for r|j^ > 0. 




FIG. 8: Probability distribution function (arbitrary units) 
for obtained as described in the text. The integral of the 
function for < r|;^ < (0.16)^ is nine-tenths of the integral for 
r'^ > 0. The vertical dashed line is drawn at — (0.16)^. 




FIG. 7: Likelihood functions as obtained from the fit de- 
scribed in the text for TZ*^.^ (solid line) and TZktt.d-/ 
(dashed line). 



In the case of decays into a D/D, there is not enough 
information to extract the ratio vb without additional 
assumptions. Thus, we first extract an upper limit on 
the experimentally measured quantity TZk-k ■ This is done 
starting from the likelihood as a function of TZktt (see 
Fig. 9) using a Bayesian method with a uniform prior 
for TZk^ > 0. The limit is TZk^ < 0.029 at 90%C.L. 
Next, in Fig. 10 we show the dependence of TZktt on 
rs, together with our limit on TZk-w- This dependence is 
shown allowing a iblcr variation on rjj, for the full range 
0° — 180° for 7 and S, as well as with the restriction 




FIG. 9: Likelihood function (arbitrary units) for TZktt as ex- 
tracted from the fit described in the text. The integral of the 
likelihood function for < TZktt < 0.029 is nine-tenths of the 
integral for TZktt > 0. The vertical dashed line is drawn at 
TZk-^ = 0.029. 



51° < 7 < 66° suggested by global CKM fits [19]. We use 
the information displayed in this Figure to set an upper 
limit on rg- The least restrictive limit on is computed 
assuming maximal destructive interference between the 
b ^ c and b ^ u amplitudes: 7 = 0°,S = 180° or 
7 = 180°, S = 0°. The limit is re < 0.23 at 90% C.L. 



14 




FIG. 10: Expectations for TZktv and the number of signal 
events vs. tb- Dark fiUed-in area: allowed region for any 
value of 5, with a ilcr variation on m, and 51° < 7 < 66°. 
Hatched area: additional allowed region with no constraint on 
7. Note that the uncertainty on ro has a very small effect on 
the size of the allowed regions. The horizontal line represents 
the 90% C.L. limit TZk^ < 0.029. The vertical dashed lines 
are drawn at rs = 0.203 and tb = 0.233. They represent the 
90% C.L. upper limits on tb with and without the constraint 
on 7. The light filled-in area represents the 68% C.L. region 
corresponding to TZktt ~ 0.013±o:oo9. 



TABLE III: Summary of the results of this analysis. 





Measured Value 


90% C.L. limit 


Tb 

^B 


o.oi3±^:;iici 
-o.oo2±[!:!jSg 
o.oii±g:Si 

o.oo4±EJ:S]S| 


< 0.029 

< 0.023 

< 0.045 

< 0.23 
< (0.16)^ 



V. SUMMARY 

In summary, we find no significant evidence for the 
decays B± ^ [K^tt^dK^ and B± ^ [K^tt^]d'K^. 



We set upper limits on the ratios Tl^^'^ of the rates for 
these modes and the favored modes [K^'k^]dK^ 
and — > [K^tt^]]j.>K^. We also use our data to set 
upper limits on the ratios of 6 — > u and b c ampUtudes 
rs and r^. All of our results are summarized in Table III. 

Our results favor values of rs and r|j somewhat smaller 
than the value of 0.2 which can be estimated from the 
measurements of color-suppressed b ^ c transitions [9]. 
If and r*g are small, the suppression of the b ^ u 
amplitude will make the determination of 7 using meth- 
ods based on the interference of the diagrams in Fig. 1 
difficult. 
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